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ABSTRACT 



This paper outlines the development of a quantitative 
instrument designed to measure students' multiple achievement goals and key 
aspects of their cognitive engagement in Australian educational settings. The 
paper demonstrates the use of congeneric measurement models in assessing and 
improving the validity of the scales comprising the Goal Orientation and 
Learning Strategies Survey (GOALS-S) , and it also illustrates the use of 
cross-validation as a way to assess the stability of the GOALS-S scales. 

Three academic goals, four social goals, three cognitive strategies, and 
three metacognitive strategies were defined and operationalized in the 
developed GOALS-S instrument. Preliminary factor analyses were used to refine 
the measure. Its 13 scales were tested with 380 Australian urban students. 
Once collected, data from a randomly chosen subsample (half the original 
sample) were subjected to confirmatory factor analysis using 13 separate 
one-factor congeneric models. Modified models that showed sufficient fit in 
the first half were tested with the second half of the sample. The initial 
modeling process supported the construct validity of only four of the 
original scales, but relatively minor modifications resulted in considerably 
better fit for the other nine scales. Results also demonstrate support for 
the reliability of at least eight of the GOALS-S scales. (Contains 6 tables 
and 88 references.) (SLD) 
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M ost educators agree that effective learning involves the ability to self-regulate a variety of 
thoughts, feelings, and actions associated with learning processes (Bouffard. Boisvert, Vezeau, 

& Larouche 1995: Hong. 1995; Meece. 1994; Schunk. 1991: Newman. 1991; Zimmerman, 1990). In 
particular., students’ ability to activate prior knowledge,, and to appropriately apply a variety of 
cognitive and metacognitive strategies in order to acquire and/or integrate new knowledge, influences 
the quality of their engagement in learning and their overall academic success (Meece. 1994, Derry, 
1990; Bransford. Vye, Kizner & Risko. 1990). Recent research has particularly focused on the ways 
students' prior knowledge, and the cognitive and metacognitive strategics students use (or do not use), 
influence their ability to acquire, integrate, and retrieve information (Hong. 1995; Zimmerman & 
Martinez-Pons. 1988). 

Various cognitive models of learning have proved useful for explaining the role that 
student's prior knowledge and strategy use plays in enhancing students’ academic learning and 
performance (Alexander & Judy. 1988; Pintrich. Cross, Kozina. & McKeachie. 1986; Weinstein & 
Mayer. 1986). Specifically, research informed by these models has been able to explain the effect of 
students’ prior knowledge and strategy' use on their perception, selective attention, encoding, retrieval, 
and problem solving abilities (Alexander, Schallert. & Hare, 1991; Winne & Marx. 1989). Moreover, 
there is growing evidence that the use of particular cognitive and metacognitive strategies is 
associated with improved learning outcomes (Paris & Winograd. 1990; Weinstein, Ridley, Dahl, & 
Weber. 1989: Brown. Bransford. Ferrara, & Campione, 1983). 

Cognitive models of learning have, however, proven less useful for explaining (a) why 
students may or may not. particularly in ‘real life' classroom situations, activate their prior knowledge 
and strategies on given learning tasks, and (b) why students fail to transfer relevant prior knowledge 
and strategies from one task or situation to another (Pintrich & Scrauben. 1992; Schneider & 
Pressley. 1989). In other words, cognitive models are less adept at explaining why students may not 
expend effort to activate and/or transfer prior knowledge and strategies. This is particularly important 
because the activation and transfer of appropriate knowledge and strategies requires effort (Carr, 
Borkowski. & Maxwell. 1991). Thus, if students do not expend effort, any knowledge and strategies 
they do posses may remain inactive (O’Niell & Douglas, 1991). 

This selective activation and transfer of prior knowledge and strategies may be attributed to 
purely cognitive factors e.g. routinisation, effective encoding, and the productive use of metacognitive 
and self-regulatory processes (Schneider & Pressley. 1989). However, recent research shows that the 
activation of prior knowledge and cognitive strategies is also dependent upon motivational variables 
(Graham & Golan, 1991; Garner. 1990; Meece. Blumenfeld, & Hoyle, 1988). Hence, students’ level 
of cognitive engagement (the extent to which students appropriately activate, transfer, and apply their 
prior knowledge "and strategies) is a function of both motivational and cognitive factors working 
together (Pintrich, 1989; Pintrich & Scrauben. 1992). This may be described as a ‘hot’ model of 
cognition (Brown et al., 1983; Gierc, 1988). 

Until recently, however, the interaction between motivational and cognitive factors in 
explaining students’ cognitive engagement and subsequent academic performance has been largely 
avoided or ignored (Corno & Mandinach, 1983; Pintrich, 1990). This is true despite a long history in 
psychology of researchers who have emphasised the motivated, purposive nature of human behaviour 
and cognition (e.g. Tolman, 1925, 1932). With some exceptions, it has generally been more common 
to explain students’ cognitive engagement in terms of either motiv ational or cognitive factors rather 
than through a combination of both. Research acknowledging the interaction between motivational 
and cognitive factors, however, may explain more completely the functioning of students’ cognitive 
processes (Borkowski, Carr, Rellinger, & Pressley. 1990; Pintrich & Schrauben, 1992). 

Despite the above comments, researchers have begun to explain cognitive engagement and 
academic performance as a product of interacting motivational and cognitive constructs (factors) (e.g. 
Hong. 1995; Pintrich, Marx & Boyle, 1993; Graham & Golan. 1991; Corno. 1986). Although the 
resurgence of this type of research is still quite recent (Pervin. 1992), there is growing empirical 
evidence that cognitive engagement is the product of interacting motivational and cognitive 
constructs. Moreover, several constructs, which are known to influence students’ motivation, also 
effect their cognitive engagement. Examples of these constructs include students self-efficacy', control 
beliefs, task value beliefs, and goal orientation (Pintrich et al., 1993). 
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Research Orientation 

Given the above, it is clearly of interest to researchers and practitioners to be able to accurately 
measure both motivational and cognitive constructs known to influence students cognitive 
engagement. The present research outlines the development of a quantitativ e instrument designed to 
measure relations between students’ goal orientations (motivational constructs) and students 
cognitive and metacognitive strategy use (cognitive constructs). 

Explanation and Justification 

The decision to measure students’ strategy use was made because, as outlined above, it appears to be a 
particularly salient indicator of students’ cognitive engagement and subsequent academic 
achievement. The decision to measure students’ goal orientation was made for reasons outlined below. 
These are preceded by a short explanation of students’ goal orientations. 

Explanation. Students’ goal orientations are cognitive representations of the different 
purposes students may adopt in achievement situations (Urdan & Maehr. 1995: Pintrich et al., 1993; 
Wentzel. 1991; Ford and Nichols, 1991). In other words, students’ goals (shorthand for their goal 
orientations) are the reasons students’ have for wanting to achieve (or not) in achievement situations. 
Typically, the achievement situations of most interest to researchers have been academic achievement 
situations. For example, students may want to achieve in academic situations in order to demonstrate 
superior performance in comparison with other students (a performance goal orientation), to 
demonstrate understanding of academic work (a mastery goal orientation), to demonstrate a 
responsible attitude (social responsibility orientation), to feel part of a group (social affiliation 
orientation), etc. Several different types of goals have been identified in the literature. By far the most 
widely researched of these are performance and mastery' goals (Ames, 1992. Blumenfeld. 1992). In 
addition to individual goals, two broader classes of goals have been identified in the literature. 
Academic goals are defined as the academic reasons students have for achieving in academic 
situations. Social goals are defined as the social reasons student have for achieving in academic 
situations. (Urdan & Maehr, 1995; Dowson & Mclnerney, 1997). 

Justification. Students’ goals have been chosen in the present study for the following 
reasons. Firstly, and most importantly, students’ goal orientation has direct effects on significant 
aspects of their cognitive engagement (Graham & Golan. 1991. Garner, 1990; Dweck & Leggett, 
1988: Nolen, 1988; Ames & Archer. 1988). Students adopting a mastery goal orientation, for 
example, are more likely to recall important information, use more and deeper cognitive strategies, 
and use more, and more adaptive, metacognitive strategies than those adopting a performance 
orientation. In addition, the entire process of academic self-regulation (of which strategy use is one 
aspect) is said to be a goal directed or goal mediated process (Newman. 1991; Kanfer & Kanfer, 
1991). Other goals may "also have an effect on specific aspects of students’ cognitive engagement. For 
example, student’s social goals have been shown to be related to their overall academic performance 
(Wentzel, 1991a. 1989). It is not unreasonable to hypothesise, therefore, that these goals have some 
effect on students’ cognitive engagement in learning. 

Secondly, there are substantial theoretical and practical links between students’ goal 
orientations and other constructs associated with their motivation and cognition. Examples of these 
constructs include self-efficacy (Schunk. 1991, 1990); self-regulated learning (Meece, 1994); intrinsic 
interest (Butler, 1987); ability perceptions (Harter, 1982); attributional beliefs (Meece, 1994) subject 
matter attitudes (Meece et al., 1988); and affect (Jagacinski & Nicholls. 1987). Moreover, students’ 
goal orientation is linked to other aspects of their academic behaviour other than their cognitive 
engagement per se. These include their task persistence (Stipek & Kowalski. 1989), their preference 
(or otherwise) for challenging or ‘risky’ learning activities (Ames & Archer, 1988; Nicholls, 1984), 
and their help-seeking behaviour (Meece, 1994; Newman, 1991). Hence, goal theory (the theoretical 
framework underlying students’ goal orientations) is both a comprehensive and complementary theory 
of academic motivation. 

Thirdly, goal theory' has clear implications for classroom practice. Several reviews (Ames, 
1992; Blumenfeld. 1992; Maehr & Midgley, 1991; Meece, 1991) have documented the ways in which 
classroom and school environments are implicated in the goals students are encouraged to adopt. 
Although empirical evidence is still restricted (see Nolen & Haladyna, 1990 for one example), it is, 
nevertheless, clear that goal theory' may direct practitioners to the value of specific classroom practices 
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designed to improve students' engagement in learning. These include providing learning 
opportunities that are meaningful and interesting (Como & Rohrkemper, 1985). providing 

opportunities for student choice and decision-making (Ryan, Connell. & Deci, 1985), promoting 
beliefs in competence through effort (Stipek & Kowalski, 1989). increasing student’s chances for 
success (Meece. 1994). rewarding and recognising personal improvement (Ames & Ames. 1991), and 
reducing the emphasis on social comparison and competition (Mac Iver. 1987). Thus, goal theory is 
not only theoretically integrated but practically applicable as well. 

Existing Instruments 

The literature contains several examples of instruments designed to measure students .: 

(a ) academic goals eg. the Motivated Strategies for Learning Questionnaire (MLSQ) 
(Pintrich. Smith. Garcia. & McKeachie. 1991) and the Inventor)- of School Motivation 
(ISM) (Mclnerenv, Roche, Mclnerney. & Marsh. 1997: Mclnernev & Sinclair. 1991). 

(b) social goals eg. the Inventory of School Motivation and the Goal Questionnaire (Wentzel, 
1989); 

(c) use of cognitive and metacognitive strategies eg. the Inventory of Learning Processes 
(ILP) (Schmeck. Geisler-Brenstein. & Cercy. 1991). the Approaches to Study Inventory 
(ASI) (Entwistle & Ramsden 1983). the Study processes Questionnaire (SPQ) (Biggs, 
1987), and the Strategic Flexibility Questionnaire (SFQ) (Cantwell, 1992). 

Moreover, several of these instruments attempt to measure combinations of the above eg. 
academic goals with learning strategies or social goals with academic goals. However, as yet. an 
instrument designed to measure all three of the above (students social goals, academic goals, and 
strategy use) is not available. This is the case despite the fact that recent research has emphasised that 
students can and do hold multiple social and academic goals in school settings and that the way 
students organise and coordinate these goals is substantially related to their cognitive engagement and 
academic performance (Seifert. 1995; Ainley, 1993. Pintrich & Garcia. 1991: Wentzel, 1991a. 1989; 
Meece. 1991). The theory underlying the instrument developed in this study explicitly assumes that 
students’ w ill hold to multiple goals and strategies simultaneously. 

In addition to the above, most of the instruments that have been designed to measure 
students’ social and academic goals have been developed outside Australia. Thus, inferences made 
concerning Australian students using scores from these instalments may be invalid. There is a need, 
therefore, to develop an instrument specifically designed in and for the Australian context from which 
valid inferences may be drawn. This is particularly so given the cultural diversity of Australian 
schools where students hold diverse sets of goals which may differentially affect their motivation and 
learning (Mclnerney, 1989a, 1989b). 

Measurement Models 

In psychological research, measurement models attempt to assess the validity of instruments i.e. the 
degree to which an instrument measures what it supposed to. They, typically, do this by assessing the 
extent to which variations in and between observed indicators are ’caused’ by underlying (or latent) 
constructs. For example, in the present study, variations in and between students’ responses to the 
GOALS-S survey questions (the observed indicators) are assumed to be ‘caused’ by students’ 
adherence to particular goals and strategies (the latent constructs the GOALS-S is attempting to 
measure). From a measurement point of view it is of most interest to know how much of the variation 
in and between the observed indicators is ‘explained’ by the latent construct(s) under consideration. If 
a substantial amount of the variation in and between a set of indicator variables can be ‘explained’ by 
the influence of a latent construct on these variables then the measurement model is ‘good’. The 
reverse is true if less than substantial amounts of variance are ‘explained’ by the latent construct. 

The degree to which variation in and between the indicator variables can be explained by a 
latent construct (or constnicts) is known as the measurement model’s ‘fit’. The most common 
measure of a model’s fit is the Chi-square (x 2 ) test which compares the degrees of freedom associated 
with a given model with the Chi-square value for the model (Hu & Bentler. 1995; Tanaka, 1993). If 
the difference between these two is small then the model is deemed to ‘fit’ (which means that the data 
are not sufficient to reject the model). If the difference is large the model does not fit (the data are 
sufficient to reject the model). The probability associated with the Chi-square value (based on its 
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degrees of freedom) acts as a standard by which the size of this difference is assessed (Hayduk, 1987; 

Cliff: 1987). , * , 

In addition to the Chi-square test, other methods for assessing model fit have been developed. 

There is considerable debate in the literature as to which of these methods (including the Chi-square 
test) are appropriate and in which situations (Tanaka. 1993; Hoyle & Panter. 1995. Marsh, Balia. & 
McDonald. 1988; Bentler & Bonett, 1980). Despite disagreement as to relative value of various 
measures of fit. however, there is a general consensus that more than one indicator of model fit should 
be used to evaluate a given model. In particular, an over-reliance on the Chi-square test alone may be 
misleading (Bentler & Bonett, 1980). 

Congeneric Measurement Models 

A common form of measurement model is the congeneric model. A congeneric model assumes the 
indicator variables used to measure the latent construct (or constructs) do so unequally well and with 
unequal variation (Joreskog, 1971). That is, the indicator variables are not all equally ‘good’ measures 
of the latent construct. A one-factor congeneric model is one that measures a single latent construct 
(although, theoretically, there is no limit on the number of indicator variables that may be used to 

measure that construct). . 

The present study uses a series of one factor congeneric models to assess how well variables 
attempting to measure each of the latent constmcts (students’ goals and strategies) actually do so. 
This is a critical step in assessing the validity of inferences drawn from the GOALS-S. If the indicator 
variables do not measure the latent constructs well then inferences drawn from the survey as to the 
relations between students’ goals and strategies will be compromised. 



Model Stability 

In addition to the overall fit of a model, researchers are, typically, interested in how stable a fitted 
model is. This is known as the model’s reliability (Cliff, 1987). A model, for example, may fit well in 
one sample but not in another. Obviously, unstable models (models that fit in one sample but not in 
others) are of limited value. A common measure of model reliability (stability) is Chronbach s alpha. 
However, for a variety of reasons this estimate is not universally appropriate, especially with 
congeneric models (Werts, Rock, Linn, Joreskog, 1978). Another particularly powerful measure of a 
model's reliability is its observed stability from one sample to another or from one portion of a sample 
to another (Browne & Cudec, 1989). Testing the fit of a model between samples or between 
independent portions of a single sample is known as cross-validation. The later type of cross- 
validation is used in the present study to test the stability of the one factor congeneric measurement 
models generated in the research. 

Purpose of the Study 

The purpose of the study was to develop an instrument capable of measuring students’ multiple social 
and academic goals in Australian educational settings. Specifically, the paper outlines: 

(a) in brief, the development of items and scales for the Goal Orientation and Learning 
Strategies Survey (GOALS-S) and, 

(b) in more detail, the testing of the GOALS-S scales (using Confirmatory Factor Analysis 
with congeneric models and cross-validation strategies) in order to establish support for 
the validity of inferences drawn from them. 

Participants 

The data represent responses to the GOALS-S from students (n=386) from schools (n=4) in the 
Sydney metropolitan region. Approximately equal numbers of male (48.8%) and female (51.2%) 
students from a cross-section of cultural, socio-economic, and academic backgrounds are represented 
in the sample. The schools were chosen from various geographical regions in the Sydney metropolitan 
area in order to maximise the socio-economic and cultural diversity of the sample. 
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Method 

A brief overview of the study follows. Elements of this overview are then discussed in more detail 
below. 

A selection of academic goals (n=3), social goals (n=4), cognitive strategies (n=3), and 
metacognitive strategies (n=3) were defined and operationalised on the basis of a previous qualitative 
studies (Dowson & Mclnerney. 1997. 1996) and a review of the literature. Following this, an initial 
item selection representing each of the goals and strategies was constructed. This item selection was 
then reviewed by a sample of students (n=6) typical of the sample with which the GOALS-S was 
eventually piloted. This was done in order to establish support for the face validity of the items. 
Several items were reworded or deleted as a result of this process. Once support for the face validity of 
the items was established, the items were ordinally scaled and the survey was piloted with a sample of 
students (described above). 

Once collected, the data from a randomly chosen sub-sample (comprising one-half of the 
original sample) were subjected to a Confirmatory Factor Analysis (CFA) process using thirteen 
separate one-factor congeneric models (discussed below). This was done in order to establish support 
for the construct validity of the scales hypothesised to be measuring particular goals and strategies 
and. hence, for the validity' of inferences drawn from them. On the basis of this initial CFA, several 
models showed substantial fit with the data. However, several others did not. With reference to both 
data and substantive theoretical issues, several items were deleted from the scales which did not in the 
original sub-sample in order to improve their fit. These modified scales were then re-tested in the 
random first-half of the sample (the same sub-sample as before). When the modified models showed 
sufficient fit in the first half of the sample, they were tested in the remaining (second) half of the 
sample. The models w hich, on the basis of the first CFA process, did not require modification were 
also tested in the second half of the data. The fit of both the modified and the un-modified models in 
the second half of the data was then assessed. 

Constructs Measures by the GOALS-S 

Table 1 below describes the goals and strategies assessed by the GOALS-S. provides a sample item 
for each, and indicates the number of items related to each goal or strategy contained in the original 
scales. 



Table 1 

Goals and Strategies measured by the GOALS-S 



Construct (Goal or Strategy) 


Sample Item 


Number of items in 
original scales 


Academic soals 
Mastery 

Wanting to achieve in order to 
demonstrate understanding, 
academic competence, or improved 
performance relative to self-established 
standards. 


I like school work that challenges me to learn new 
things. 


5 


Performance 

Wanting to achieve in order to out 
perform other students, attain certain 
grades /marks, or obtain tangible rewards 
associated with academic performance. 


I want to get better marks than other people at school. 


6 


Work avoidance 

Wanting to achieve with as little effort as 
possible. Conversely, avoiding demanding 
achievement situations. 


1 choose easy work at school so that 1 don't have too 
much trouble with it. 


5 




6 7 
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Construct (Goal or Strategy) 


Sample Item 


Number of items in 
original scales 


Social zoals 
Social affiliation 

Wanting to achieve in order to enhance a 
sense of belonging to a group or groups 
and/or to build or maintain inter-personal 
relationships. Conversely, wanting to 
achieve in order to avoiding feelings of 
separateness or isolation. 


I try to work with my friends as much as possible at 
school 


5 


Social approval 

Wanting to achieve in order to gain the 
approval of peers, teachers, and/or 
parents. Conversely, wanting to achieve in 
order to avoid social disapproval or 
rejection. 


Getting praise from my teachers for my school work is 
important to me 


5 


Social responsibility 
Wanting to achieve in order to maintain 
interpersonal commitments, meet social 
role obligations, or follow social and 
moral ‘rules’. Conversely, wanting to 
achieve in order to avoiding social 
transgressions and/or unethical conduct. 


It is important for students to help each other at school. 


5 


Social status 

Wanting to achieve in order to attain 
wealth and or position in school and^r 
later life. Conversely, wanting to achieve 
in order to avoid low status positions in 
either school or later life. 


I want to do well at school for the good of my family’s 
reputation. 


5 


Cognitive strategies 
Elaboration 

Making connections between present and 
previously learned information. May 
involve paraplirasing, generating 
analogies, reviewing previous work, etc. 


When I want to learn new things I try to recall what I 
know about similar things. 


4 


Organisation 

Selecting, sequencing, outlining, re- 
ordering or summarising important 
information. 


I make summaries of the things I want to learn at 
school. 


5 


Rehearsal 

Listing, memorising, reciting, and/or 
naming facts/items to be learned. 


When I want to learn something for school I practice 
saying it over and over. 


5 


A fetacoenitive strategies 
Monitoring 

Involves self-checking for understanding, 
self-testing, organised review of learned 
material, etc. Implies systematic attempt 
to evaluate the assimilation and 
organisation of learned material. 


I ask myself questions to see if I understand what I am 
learning 


4 


Planning 

Involves prioritising, time management, 
scheduling, setting realistic goals, 
arranging work environments 
appropriately, etc. Implies thoughtful 
preparation for completing work. 


When I want to leam things for school I try to pick out 
the most important parts first. 


5 


Regulating 

The strategies put in place to rectify 
deficits identified whilst monitoring. 
Specific strategies include attempting 
different ways to learn material, seeking 
explanations from teachers, identify ing 
mistakes in reasoning, etc. 


If I don't understand something I will try a different 
way to leam it. 


5 
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Initial Factor Analyses 

Alter listwise deletion there were 380 cases available for analysis. Initially, the original one-factor 
congeneric models were tested with a randomly chosen sub-sample of the data comprising one-half of 
the original sample (n=190). That is. each of the goals and strategies outlined above were the latent 
factor in a series of models where the survey items supposedly measuring those constructs were the 
indicator variables. There were thirteen models in total. PRELIS (version 2.1) and LISREL (version 
7) (Joreskog & Sorbom. 1987) embedded in SPSS-X (version 6.1) were used to complete the analyses. 

As a result of these analyses, several models were deemed not to fit the data sufficiently. In 
order to improve the fit of these models, items which did not appear to measure the latent construct 
well were deleted from the original scales. The rationale for these deletions was as follows. Firstly., the 
absolute fit of the models was assessed on the basis of their Chi-square/degrees of freedom ratio and 
associated probabilities. Secondly, the relative fit of the models was assessed on the basis of other fit 
indices provided by LISREL i.e. the Goodness of Fit Index (GFI), the Adjusted Goodness of Fit Index 
(AGFI) and the Root Mean Square Residual (RMSR). The GFI and AGFI should be as close as 
possible to one (1). The RMSR should, ideally, be less than .05. 

If the overall fit of the model was questionable on the combined basis of these absolute and 
relative indicators, then, particularly, the factor loadings (the degree to which indicator variables are 
associated with the latent constructs) for each item constituting the original scales were more closely 
examined. Items which had low factor loadings were considered candidates for deletion from a scale. 
However, a low factor loading per se did not mean that an item was automatically deleted from a 
scale. On some occasions, for example, an item was considered central to the scale being constructed 
and was retained despite its apparently low factor loading. Conversely, items with moderate factor 
loadings were not necessarily ‘safe’ from deletion. Again, on several occasions, items with moderate 
factor loadings were removed from scales because they appeared to be negatively influencing the 
theoretical integrity' of the scale. Thus, the decision to delete or not or delete was based upon 
substantive theoretical grounds as well as data considerations. 

Subsequent Factor Analyses 

Once ‘suspect’ items had been deleted from the original scales the modified scales were retested in the 
same random sub-sample as before. Again, the absolute and relative fit of these models as well as the 
factor loadings for each item in the scales were assessed. 

It should be noted, however, that despite the best intentions of the researcher, the process 
above does ‘capitalise on chance’. That is, by testing and retesting models in a given sample the 
researcher increases the chances of finding model that fits the data regardless of the considerations 
that informed the model modification process (MacCallum, Roznowski. Necovvitz. 1992; Cliff, 1983). 
Thus, it is important to test any modified model once and once only in an independent sample or sub- 
sample. In the present research, once the modified models demonstrated sufficient fit, they were 
tested in the remaining (as yet untested) half of the original sample. This was done once only. The fit 
of the models in the second half of the data was then examined. 

Results 



The results of the first series of analyses (testing the original thirteen models in the random first-half 
of the data) are reported in Table 2. The results in Table 2 indicate the following. First, four (4) of the 
original scales (Mastery, Rehearsal, Elaboration, and Monitoring) model the data well. That is, both 
the absolute (Chi-square) and relative (GFI. AGFI, RMSR) fit indices do not suggest that the data are 
sufficient to reject these models. These models, thus, were left unaltered and were tested later in the 
second half of the data (described below). 

Second, seven (7) of the models (Performance, Work Avoidance, Social Affiliation, Social 
Status. Social Approval, Social Responsibility, and Regulation) appear to model the data poorly. That 
is, the Chi-square test and relative fit indices indicate that the data are sufficient to reject these 
models. Third, two (2) of the models (Organisation, Planning) demonstrate appropriate fit according 
to the Chi-square test, but inappropriate fit according to, at least, the RMSR. These models also have 
one or more items that ‘load’ weakly on the underlying factor measured by the models. Thus, 
although these models fit according to the Chi-square test there is reason to suspect that the models do 
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Table 2 

Thirteen Original One Factor Congeneric Models of Students’ Goals and Strategies 

(Random First-Half of Data) 



Factor 


Number of 
Indicators 


Factor 

Loadings 


Chi- 

square 


Degrees 

Freedom 


Probability 


GFI 


AGFI 


R\ISR 


Mastery 


5 


.682 

.576 

.816 

.846 

.772 


7.71 


5 


.173 


.984 


.953 


.028 


Performance 


6 


.812 

.741 

.596 

.882 

.574 

.594 


37.32 


9 


.000 


.934 


.845 


.061 


Work 

Avoidance 


5 


.719 

.811 

.443 

.400 

.210 


10.89 


5 


.054 


.978 


.934 


.052 


Social 

Affiliation 


5 


.789 

.615 

.878 

.239 

.662 


13.51 


5 


.019 


.972 


.917 


.040 


Social Approval 


5 


.658 

.722 

.722 

.510 

.481 


16.14 


5 


.006 


.965 


.895 


.048 


Social 

Responsibility 


5 


.132 

.323 

.894 

.747 

.785 


34.10 


5 


.000 


.938 


.815 


.093 


Social Status 


5 


.780 

.373 

.708 

.892 

.685 


17.80 


5 


.003 


.965 


.895 


.050 


Elaboration 


4 


.592 

.445 

.958 

.449 


.98 


2 


.612 


.997 


.987 


.016 


Organisation 


5 


.029 

.617 

.747 

.153 

.277 


10.27 


5 


.068 


.979 


.937 


.054 


Rehearsal 


5 


.495 

.338 

.571 

.598 

.176 


7.07 


5 


.216 


.985 


.956 


.041 


Monitoring 


4 


.582 

.562 

.433 

.414 


.75 


2 


.688 


.998 


.990 


.015 


Planning 


5 


.161 

.178 

.512 

.970 

.428 


10.11 


5 


.072 


.980 


.939 


.050 


Regulation 


5 


.156 

.341 

.644 

.498 

-.006 


.17.23 


5 


.004 


.962 


.887 


.074 
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not. in fact, fit the data well (especially given the relatively small sample size which may artificially 
reduce the Chi-square value relative to its degrees of freedom). 

Given the above, several indicator variables in the seven poorly fitting and two indifferently 
fitting models were removed from the models according to the process outlined previously. These 
nine (9) modified models were then re-analysed in the same half of the data as the original models. 
Results of these analyses are reported in Table 3. 

Table 3 

Nine Modified One Factor Congeneric Models of Students’ Goals and Strategies 

(Random First-Half of Data) 



Factor 


Number of 
Indicators 


Factor 

Loadings 


Chi- 

square 


Degrees 

Freedom 


Probability’ 


GFI 


AGFI 


RMSR 


Performance 


4 


.818 

.818 

.467 

.876 


.46 


2 


.795 


.999 


.994 


.008 


Work 

Avoidance 


4 


.786 

.747 

.465 

.342 


5.15 


2 


.054 


.987 


.933 


.041 


Social 

Affiliation 


4 


.654 

.690 

.284 

.733 


1.15 


2 


.563 


.997 


.985 


.016 


Social Approval 


4 


.655 

.691 

.804 

.568 


5.27 


2 


.072 


.986 


.929 


.031 


Social 

Responsibility 


4 


.314 

.855 

.769 

.811 


2.44 


2 


.295 


.994 


.968 


.020 


Social Status 


4 


.744 

.657 

.895 

.677 


4.65 


2 


.098 


.988 


.941 


.024 


Organisation 


4 


.561 

.807 

.190 

.248 


4.25 


2 


.119 


.989 


.946 


.042 


Planning 


4 


.189 

.408 

.600 

.213 


.70 


2 


.703 


.998 


.991 


.018 


Regulation 


4 


.333 

.876 

.299 

.220 


.87 


2 


.646 


.998 


.988 


.019 



The results in Table 3 indicate that the data in the random first-half of the sample are 
insufficient to reject the nine modified models. However, as above, several of the indicator variables 
retain low factor loadings. This suggests that, although the nine modified models fit in the first half of 
the data, they may be unreliable (unstable). Whatever the case, the next step was to test the nine 
modified and four original un-modified models in the second half of the data. The results of these 
analyses are reported in Table 4. 

The analyses in Table 4 indicate that all nine of the modified models as well as the four 
original (unaltered) models fit the data in the second half of the sample. Thus, the data in the second 
half of the sample are insufficient to reject these models. However, again, low factor loadings are 
retained by some indicator variables. This may compromise the reliability of the scales despite their 
apparent construct validity. 
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Table 4 

Nine Modified and Four Un-modified One Factor Congeneric Models of Students’ 

Goals and Strategies 
(Random Second Half of Sample) 



Factor 


Number of 
Indicators 


Factor 

Loadings 


Chi- 

sqnare 


Degrees 

of 

Freedom 


Probability 


GFI 


AG FI 


RMSR 


Mastery 


5 


.630 

.699 

784 

805 

.750 


5.15 


5 


.398 


.990 


.970 


.020 


Performance 


4 


.836 

.755 

.599 

.844 


4.52 


2 


.104 


.989 


.944 


.021 


Work 

Avoidance 


4 


.831 

.731 

.516 

.390 


4.51 


2 


.105 


.988 


.942 


.035 


Social 

Affiliation 


4 


.610 

.722 

.288 

.690 


1.23 


2 


.542 


.997 


.984 


.018 


Social Approval 


4 


.655 

.682 

.755 

.510 


3.69 


2 


.158 


.990 


.951 


.028 


Social 

Responsibility 


4 


.319 

.882 

.799 

.719 


4.34 


2 


.114 


.988 


.942 


.027 


Social Status 


4 


.762 

.701 

.872 

.722 


.26 


2 


.876 


.999 


.997 


.005 


Elaboration 


4 


.642 

.261 

.891 

.423 


.96 


2 


.708 


.998 


.991 


.013 


Organisation 


4 


.550 

.798 

.179 

.200 


2.10 


2 


.351 


.994 


972 


.026 


Rehearsal 


4 


.317 

.503 

.730 

.423 


1.75 


2 


.417 


.996 


.978 


.023 


Planning 


4 


.238 

.400 

.609 

.260 


1.04 


2 


.594 


.997 


.986 


.022 


Monitoring 


4 


.662 

.526 

.274 

.380 


2.39 


2 


.303 


.994 


.969 


.030 


Regulation 


4 


.337 

.938 

.297 

.230 


.46 


2 


.794 


.999 


.994 


.014 



The final step in the analyses was to assess the fit of the thirteen models (above) in the entire 
data set. This was done in order to establish final parameter estimates and measures of fit for all the 
available data in the sample. Results of these analyses are reported in Table 5. The results in Table 5 
indicate that all the models fit with the entire data set. However, according to the Chi-square test, the 
work avoidance model only ‘just’ fits the data. That is, the data according to this test are almost 
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sufficient to reject this model. Also, consistent with the previous analyses, several of the models 
display items with weak factor loadings. In order to better quantify the effect that these low factor 

Table 5 

Nine Modified and Four Un-modified One Factor Congeneric Models of Students’ 

Goals and Strategies 
(Complete Sample) 



Factor 


Number of 
Indicators 


Factor 

Loadings 


Chi - 

square 


Degrees 

Freedom 


Probability 


GFI 


AG FI 


RMSR 


Mastery 


5 


.649 

.582 

.754 

.817 

.722 


4.49 


5 


.481 


.991 


.973 


.021 


Performance 


4 


.803 

.763 

.508 

.861 


.71 


2 


.701 


.998 


.991 


.009 


Work 

Avoidance 


4 


.785 

.825 

.482 

.447 


5.97 


2 


.051 


.992 


.961 


.029 


Social 

Affiliation 


4 


.620 

.715 

.266 

.710 


2.18 


2 


.337 


.997 


.986 


.016 


Social Approval 


4 


.536 

.708 

.751 

.599 


5.46 


2 


.065 


.993 


.964 


.023 


Social 

Responsibility 


5 


.228 

.789 

.812 

.806 


3.59 


2 


.122 


990 


.952 


.025 


Social Status 


4 


.742 

.676 

.904 

.718 


1.46 


2 


.482 


.998 


.991 


.009 


Elaboration 


4 


.540 

.349 

.949 

.468 


.92 


2 


.932 


.998 


.991 


.017 


Organisation 


4 


.327 

.709 

.173 

.183 


1.76 


2 


.415 


998 


988 


.020 


Rehearsal 


4 


.320 

.424 

.692 

.517 


4.53 


2 


.104 


.994 


.970 


.028 


Monitoring 


4 


.602 

.523 

.295 

.478 


3.99 


2 


.136 


.995 


.974 


.025 


Planning 


4 


.171 

.648 

.747 

.208 


1.89 


2 


.390 


.998 


.988 


.021 


Regulation 


5 


.248 

.989 

.228 

.189 


.65 


2 


.724 


.999 


.996 


.012 




loadings may have on the reliability of each of the scales, the coefficients of determination 
(maximised reliabilities) for each of the scales are reported in Table 6 below. 
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Table 6 

Maximised Reliabilities for the GOALS-S Scales 



Construct 


Maximised Reliability 


Masterv 


.850 


Performance 


.865 


Work Avoidance 


.811 


Social Affiliation 


.735 


Social Approval 


.766 


Social Responsibility 


.846 


Social Status 


.884 


Elaboration 


.908 


Organisation 


.545 


Rehearsal 


.620 


Monitoring 


.572 


Planning 


.673 


Regulation 


.979 



These results indicate that four of the scales (Organisation, Rehearsal, Monitoring, and 
Planning) have relatively low maximised reliabilities. 

Discussion 

The process outlined above provides some support for the construct validity of the GOALS-S scales. 
The initial modeling process supported the construct validity of only four of the original thirteen 
scales. This result was not totally unexpected given that the GOALS-S scales were previously 
untested. However, the fact that relatively minor modifications to the nine non-fitting scales resulted 
in considerably better fit for these scales suggests that their fit was not as 4 bad' as. might be suggested 
from the initial analyses. Whatever the case, these modified scales did fit the random first-half of the 
data sufficiently. This was tme despite the fact that, as noted previously, some of the factor loadings 
on some of the scales did not appear substantial. (One of the deficiencies in using one factor 
congeneric models is that the possibility that one item might load’ on more than one factor is not 
assessable. Therefore there is no way of testing whether items with low (or high or moderate) factor 
loadings may ‘load’ on other factors or not). 

Despite this, the nine modified models, as well as the four unaltered models, fitted the data in 
the second half of the sample adequately. This cross-validation is a substantial test of the stability of 
the models, despite some low factor loadings. Furthermore, confirmation of the construct validity and 
(to some extent) the reliability of the scales comes from the test of the modified and unmodified 
models in the entire data set. All the models continued to display adequate fit within the complete 
data set. 

Although not an exception to the above, it is interesting to examine the fit of the Work 
Avoidance model in the complete data set. As indicated above, this model, according to the Chi- 
square test, fails only marginally to be rejected by the complete data set. This may be indicate that the 
model only marginally fits the data. Alternatively, it may indicate a well noted deficiency of the Chi- 
square test i.e. that it is sample size dependent. Typically, as the sample size increases the Chi-square 
value relative to its degrees of freedom increases. This means that, in large sample sizes, the Chi- 
square/degrees of freedom ratio may be statistically significant even though the model substantially 
fits the data. While the ‘excuse' that sample size has contributed to a statistically significant Chi- 
square value is overused (Hayduk, 1987), it is, nevertheless, possible that the Chi-square value 
associated with the Work Avoidance model has been inflated relative to its degrees of freedom as a 
result of the increase in sample size from one-half to the complete sample. Whatever the case, none 
of the fit indices (GFI, AGF1, RMSR) suggest that the fit of this model is as marginal as the Chi- 
square probability indicates. 

Finally, the reliability estimates in Table 6 suggest that four of the GOALS-S scales may be 
unstable. It should be noted that the reliability coefficients are estimates only and that, in the present 
study, the cross validation strategy used confirmed that the scales were stable from one-half of the 
data to the other and in the complete data set. Nevertheless, the relatively low reliability estimates 
suggest that, in other samples, these models may not be stable. Further testing in other sample will be 
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necessary to confirm whether all the scales, but particularly the scales with low reliabilities, are 
unstable. 



Conclusion 

This study has demonstrated the use of one-factor congeneric models and cross-validation techniques 
in the development of the GOALS-S. a quantitative instrument designed to measure students' social 
and academic goals and cognitive and metacognitive strategy use. The analyses above support the 
construct validity of the GOALS-S scales. Moreover, they demonstrate support for the reliability of at 
least eight of the GOALS-S scales. Given these results, the study provides initial support for 
inferences which may be drawn from GOALS-S. However, there is clearly the need to re-test the 
GOALS-S scales with other samples to further establish (or not) their constnict validity and reliability 
and. hence, the validity 7 of inferences drawn from them. 
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